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ABSTRACT 

Background Surgical complications represent a 
significant cause of morbidity and mortality with 
the rate of major complications after inpatient 
surgery estimated at 3-17% in industrialised 
countries. The purpose of this review was to 
summarise experience with surgical checklist use 
and efficacy for improving patient safety. 
Methods A search of four databases (MEDLINE, 
CINAHL, EMFJASE and the Cochrane Database of 
Controlled Trials) was conducted from 1 January 
2000 to 26 October 2012. Articles describing 
actual use of the WHO checklist, the Surgical 
Patient Safety System (SURPASS) checklist, a 
wrong-site surgery checklist or an anaesthesia 
equipment checklist were eligible for inclusion 
(this manuscript summarises all but the 
anaesthesia equipment checklists, which are 
described in the Agency for Healthcare Research 
and Quality publication). 
Results We included a total of 33 studies. We 
report a variety of outcomes including avoidance 
of adverse events, facilitators and barriers to 
implementation. Checklists have been adopted in 
a wide variety of settings and represent a 
promising strategy for improving the culture of 
patient safety and perioperative care in a wide 
variety of settings. Surgical checklists were 
associated with increased detection of potential 
safety hazards, decreased surgical complications 
and improved communication among operating 
staff. Strategies for successful checklist 
implementation included enlisting institutional 
leaders as local champions, incorporating staff 
feedback for checklist adaptation and avoiding 
redundancies with existing systems for collecting 
information. 

Conclusions Surgical checklists represent a 
relatively simple and promising strategy for 
addressing surgical patient safety worldwide. 
Further studies are needed to evaluate to what 
degree checklists improve clinical outcomes and 
whether improvements may be more 
pronounced in particular settings. 



THE PROBLEM 

Although surgery represents a mainstay of 
medical treatment, in industrialised coun- 
tries, the rate of perioperative death dir- 
ectly due to inpatient surgery has been 
estimated at 0.4-0.8%, and the rate of 
major complications has been estimated at 
3-17%. 1 2 These complications include 
wrong patient/procedure/site surgery, 
anaesthesia equipment problems, lack of 
availability of necessary equipment, 
unanticipated blood loss, non-sterile 
equipment, and surgical items (eg, 
sponges) left inside patients. The com- 
plexity of most surgical procedures 
requires a well coordinated team to 
prevent these events. 

STRATEGIES FOR PATIENT SAFETY 

Surgical checklists can potentially prevent 
errors and complications which may 
occur during surgery or perioperatively. 
A variety of interventions have shown 
promise for improving patient safety. For 
instance, Neily et al 3 found that surgical 
team training which incorporated surgical 
checklists along with communication 
strategies was associated with a significant 
reduction in surgical mortality. Arriaga 
et al 4 found that checklists dramatically 
improved adherence to critical processes 
of care in simulated scenarios of surgical 
crises. Studies have suggested that check- 
lists may reduce errors for many reasons, 
including ensuring that all critical tasks 
are carried out, encouraging a non- 
hierarchical team-based approach, enhan- 
cing communication, catching near 
misses early, anticipating potential com- 
plications, and having technologies to 
manage anticipated and unanticipated 
complications. The WHO Surgical Safety 
Checklist is a prominent example of a 
surgical checklist intended to ensure safe 
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surgery and minimise complications. Launched in 
June 2008, it has been translated into at least six lan- 
guages. 5 The 2009 WHO checklist (http://www.who. 
int/patientsafety/safesurgery/en/) contains 22 items in 
three phases: 

► Before induction of anaesthesia, covering areas such as 
patient identification, anaesthesia equipment check and a 
pulse oximetry check. 

► Before skin incision, covering areas such as team intro- 
ductions, review of critical steps and antibiotic 
prophylaxis. 

► Before patient leaves operating room (OR), covering 
areas such as checking counts of instruments, specimen 
labelling and concerns for recovery. 

In this paper we discuss the evidence for three 
patient safety efforts associated with surgical check- 
lists. The WHO Surgical Safety Checklist and the 
Joint Commission Universal Protocol (UP) for 
Preventing Wrong Site, Wrong Procedure, Wrong 
Person Surgery have each been widely implemented 
to improve care when surgical procedures are per- 
formed. We also discuss the Surgical Patient Safety 
System (SURPASS) checklist, 7-10 which represents a 
more comprehensive approach, capturing clinical care 
from admission to surgery to discharge. 

REVIEW STRATEGY 

We conducted a systematic literature search of 
MEDLINE, CINAHL, EMBASE and the Cochrane 
Database of Controlled Trials using a search strategy 
developed by a medical librarian. The search strategy 
(available upon request) included studies published 
from 1 January 2000 to 26 October 2012, and used a 
combination of medical subject headings and key- 
words related to checklists ('anaesthesia checklist', 
briefing, checklist, checkout, communication, docu- 
mentation, instrument, 'safety checklist', tool, 'surgical 
checklist', protocol, 'WHO checklist'). 

Given the limited scope of this review, we focused 
on any articles describing actual use of the WHO 



checklist, the SURPASS checklist, a wrong-site surgery 
checklist or anaesthesia equipment checklists. We rec- 
ognise other surgical checklists exist; however, many 
of these have only been implemented at a single insti- 
tution. We also included articles describing use of 
anaesthesia checklists to detect equipment failure in 
simulated scenarios. This manuscript summarises all 
but the anaesthesia equipment checklists, which are 
described in an Agency for Healthcare Research and 
Quality (AHRQ) publication. 11 An overview of the 
three types of checklists discussed in this paper is 
given in table 1. We included a total of 33 studies of 
these checklists and tabulated the reported outcomes, 
facilitators and barriers to checklist implementation. 

BENEFITS AND HARMS 

Benefits 

WHO checklist 

The 2008 WHO Surgical Safety Checklist was tested 
at eight sites around the world. 5 These settings varied 
greatly in the number of beds (range 371-1800), the 
number of ORs (range 3-39), and the income level of 
the country (four low, four high). Surgical safety pol- 
icies prior to implementation of the WHO Checklist 
also differed regarding the use of routine intraopera- 
tive monitoring with pulse oximetry (six of eight 
sites), oral confirmation of patients' identity and surgi- 
cal site in the OR (only two of eight sites), and 
routine administration of prophylactic antibiotics in 
the OR (five of eight sites). None of the eight sites 
had a 'standard plan for intravenous access for cases 
of high blood loss', or formal team briefings preopera- 
tively or postoperatively. 

Baseline data were obtained at each site for 
3 months prior to checklist introduction, involving a 
total of 3733 surgical procedures. In the subsequent 
3-6-month period after checklist introduction, involv- 
ing 3955 procedures, data showed decreases in patient 
mortality (from 1.5% to 0.8%) and inpatient compli- 
cations (from 11% to 7%). No single site was driving 



Table 1 Overview of the three checklists 



Checklist Clinical scope Staff involvement Categories and numbers of items 



WHO Surgical 
Safety Checklist 


Surgical care 


Surgeon(s), anaesthetist(s), 
nurse(s) 


Total of 22 items, in three categories: 

► Before induction of anaesthesia 
(7 items) 

► Before skin incision (10 items) 

► Before patient leaves operating 
room (5 items) 


SURPASS 


All surgical care between patient admission and 
discharge 


Ward doctor(s), surgeon(s), 
anaesthetist(s), nurse(s) or 
operating assistant(s) 


Total of 90 items, in 1 1 categories 


Checklists based on 
the Universal 
Protocol 


Surgical care, but also (if applicable) when the 
procedure is scheduled, when the patient enters the 
healthcare facility, and anytime care is transferred 
between caregivers 


Varies by site 


Varies by site 



SURPASS, Surgical Patient Safety System. 
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the findings, as evidenced by the persistence of find- 
ings after the removal of any single site in a sensitivity 
analysis. The authors found that the performance 
rates for six specific safety indicators (eg, using a 
pulse oximeter) also increased after checklist introduc- 
tion, suggesting that the safety indicators may have 
been responsible for the lower rates. 

In discussing the results, the authors acknowledged 
that the underlying explanations were 'most likely 
multifactorial' and included the following: 

► The checklist itself. 

► A Hawthorne effect (ie, rates may have decreased 
because OR personnel knew they were being measured). 
The authors argued against this possibility based on two 
aspects of their data: this knowledge was in place before 
and after checklist introduction, and the subset of proce- 
dures for which study personnel were present in the OR 
had the same reductions in complications as procedures 
when study personnel were absent from the OR. 

► The simple existence of a formal pause or preoperative 
briefing (which could be done without a 'checklist'). 
Such a pause is a necessary component of the checklist. 

► Increased uptake of safety technologies (eg, administer- 
ing antibiotics in the OR rather than in preoperative 
wards). This change could be considered a byproduct of 
checklist introduction (ie, hospitals made more antibio- 
tics directly available in the OR because of the presence 
of an antibiotics-related item on the checklist). 

► A broad change in safety culture and teamwork at that 
site, an explanation supported by the finding that greater 
increases in safety attitudes at the pilot sites were asso- 
ciated with greater reductions in complications. 12 

Subsequent publications about the WHO Surgical 
Safety Checklist have found improvements in urgent 
surgery 13 and safety attitudes. 12 14 Haynes et al 12 
reported that 80% of respondents considered the 
checklist easy to use, 20% believed it took too long 
and 93% of respondents would want the checklist 
used if they were undergoing surgery. Likewise, 
Helmio and colleagues 15 found that 76% of OR staff 
agreed the checklist improved safety, 68% agreed it 
improved error prevention and 93% would want the 
checklist used if they were having surgery. Team 
members reported high satisfaction and positivity 
about the checklist, and estimated that it only took 
about 2 min to complete. 16 

SURPASS checklist 

The WHO checklist focuses primarily on events 
occurring within the OR. However, an estimated 53- 
70% of surgical errors occur outside the OR. 8 17 18 
The SURPASS checklist 7 " 10 attempts to address these 
errors by encompassing all care between patient 
admission and discharge. Within the OR itself, the 
SURPASS checklist is less specific than the WHO 
checklist (eg, the SURPASS checklist does not specific- 
ally mention any of the following: pulse oximetry, dif- 
ficult airway, risk of blood loss (although it asks 



whether blood products are available), team introduc- 
tions, and anticipation of critical events). 

De Vries et al 7 tested the 90-item SURPASS check- 
list. In six test hospitals, the 3 -month period after the 
checklist was initiated (compared with the 3 months 
before) saw numerous improvements: decreases in the 
percentage of patients with complications, in-hospital 
mortality, patient temporary disability and reopera- 
tions. No such improvements were found among the 
five control hospitals. Interestingly, the degree of 
improvement was associated with greater compliance 
with the checklist, providing greater confidence that 
the checklist itself was responsible for improvements. 
A subsequent retrospective review of 294 medical 
claims 10 estimated that 40% of deaths and 29% of 
liability incidents might have been prevented if the 
SURPASS checklist had been used. Further review of 
6313 checklists performed found that 41% detected 
at least one oversight, with the most common occur- 
ring postoperatively (lack of postoperative instructions 
concerning ventilation by the anaesthesiologist and 
missing medication prescriptions at discharge). 19 

Wrong-site surgery checklists 

In January 2004, the Joint Commission launched the 
first version of the UP for Preventing Wrong Site, 
Wrong Procedure, Wrong Person Surgery. 6 20 
Preoperative verifications of person, procedure and 
site are supposed to occur in the OR and (if applic- 
able) when the procedure is scheduled, when the 
patient enters the healthcare facility, and anytime care 
is transferred between caregivers. Site marking should 
involve only the operative site and should be visible 
before the patient is draped. The 'time out' is to occur 
before incision and involve the entire OR team. The 
UP is not a checklist 21 but could be implemented 
using one or more checklists. Steps 1 and 3 specific- 
ally mention the potential use of a checklist. 

Wrong-site surgery is rare; estimates for various pro- 
cedures range from 1 in 13 000 procedures for 
wrong-site anaesthesia block to 1 in 4200 for wrong- 
side ureteral stents. 22 A general systematic review esti- 
mated that the overall rate was 1-5 per 10 000 proce- 
dures. 23 Given the rarity, demonstrating a statistical 
reduction would require an unfeasibly large study. A 
systematic review searched for literature and con- 
cluded there was 'no literature to substantiate the 
effectiveness of the current Joint Commission 
Universal Protocol in decreasing the rate of wrong 
site, wrong level surgery.' 23 Therefore, the preventive 
benefits of a checklist to prevent wrong-site surgery 
are generally assumed based on clinical expertise. 

HARMS 

Direct harms of surgical checklists have not been 
reported. In 2011, Sewell et al 24 reported that after 
WHO implementation, the rate of lower respiratory 
tract infections actually increased from 2.1% to 2.5%. 
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Whether this increase was caused by the checklist is 
unclear; however the authors attributed rate reduc- 
tions to the checklist, so they could also have attribu- 
ted rate increases to the checklist. Despite the absence 
of reported direct harms, some checklist users have 
expressed concern regarding potential harms. For 
instance, some worry that checklist use decreases OR 
efficiency or creates unnecessary patient anxiety. In 
2011, Kearns et al 25 reported that 3 months after 
WHO checklist implementation, 30% believed it was 
an inconvenience in emergency cases; however, this 
percentage was lower than it had been prior to imple- 
mentation of the checklist when staff were asked 
hypothetically whether they believed it would be an 
inconvenience in emergency cases (53% said it would 
be). OR efficiency might also be compromised if 
checklists duplicated already existing safety procedures 
or if nurses responsible for performing the checklist 
were unfamiliar with its execution due to high staffing 
turnover. 26 27 In one study, 27 staff expressed concerns 
that prompting patients for their name several times 
immediately before induction of anaesthesia might 
create unnecessary anxiety. 

IMPLEMENTATION CONSIDERATIONS AND COSTS 
WHO checklist 

We included 23 reports of WHO checklist implemen- 
tation. Twenty-one studies reported WHO checklist 
implementation at other sites and two reported 
experience at institutions involved in the original 
study (table 2). 

Results from the 23 implementation reports appear 
in table 3. In keeping with WHO recommendations, 
checklists were tailored and implemented differently 
for a wide variety of contexts. At present, it remains 
unclear whether OR posters, paper tick boxes or elec- 
tronic medical records perform better. Feedback from 
surgical teams was generally positive, but support 
tended to be greater from nurses and anaesthetists 
than from surgeons. For example, Vats et al found 
that anaesthetists and nurses were 'largely supportive' 
but some surgeons were 'not very enthusiastic'. 

Reasons cited for success included good training 
and staff understanding, a local champion, support 
from upper management, being able to modify the 
checklist, distribution of responsibility, the feeling of 
ownership by team members, a stepwise implementa- 
tion process which incorporated real-time feedback, 
and enhanced communication and teamwork. 
Regarding communication, for example, Sewell et al 24 
found that 77% of users thought the checklist 
improved team communication; this percentage was 
70% in the study by Kearns et al. 25 The implementa- 
tion study by Conley et al 28 emphasised that the local 
champion should 'persuasively explain why and adap- 
tively show how to use the checklist'. Styer et al 29 and 
Bohmer et al 30 attributed success to recruiting senior 
leaders of their institutions to be local champions and 



incorporating real-time feedback into checklist 
protocols. 

Barriers to implementation generally fell into four 
categories: confusion regarding how to properly use 
the checklist, pragmatic challenges to efficient work- 
flow, access to resources, and individual beliefs and 
attitudes. First, OR staff were sometimes confused 
about how to properly execute the checklist. 15 27 31 
For instance, Levy et al 31 found significant confusion 
about the timing of checklist items and who was 
responsible for prompting checklist questions among 
OR staff. While inadequate education may play a part, 
Fourcade et al 27 found that nurses were unfamiliar 
with the checklist because of high staffing turnover. 
Vogts et al 32 suggested that performance of 'sign out' 
may be low since this section is not linked to a specific 
event in patient management, unlike the 'sign in' and 
'time out' domains and thus lacks clarity. 

Second, checklist implementation occasionally 
created pragmatic problems for OR workflow. 
Particular challenges include extra time, 27 32 especially 
during emergency procedures, 33 and duplication of 
safety checks already routinely performed. 26 27 In the 
study by Kearns et al 25 3 0% felt that in emergency 
cases, the checklist was inconvenient. Third, develop- 
ing countries often lacked regular access to resources. 
Yuan et al i4 reported that inconsistent access to anti- 
biotics and batteries hampered checklist use in two 
Liberian hospitals. Likewise, Kasatpibal et al 34 
reported that surgical sites were not routinely marked 
because marking materials were unavailable in a Thai 
hospital. Finally, individual attitudes of staff towards 
the checklist played a major role in the outcome of 
implementation. Barriers included general surgeon 
resistance to changing habits, awkwardness of self- 
introductions and steep interpersonal hierarchy. Some 
nurses reported concerns about incurring legal 
responsibility if a complication occurred after they 
signed the checklist form. 



Health outcomes 

In terms of improved health outcomes (rightmost 
columns of table 3), 10 of the 21 implementation 
studies reported relevant data. Among the 10 report- 
ing studies, however, reductions were generally 
impressive. For example, Askarian et al 35 found that 
surgical complications decreased from 22.9% to 10%. 
Yuan et al 14 reported that two Liberian hospitals 
found checklist introduction was significantly asso- 
ciated with fewer surgical site infections (adjusted OR 
(AOR) 0.28; 95% CI 0.15 to 0.54) and surgical com- 
plications (AOR 0.45; 95% CI 0.26 to 0.78). 

Similarly, the study at Royal Bolton 36 found that 
nine potential safety incidents were averted during a 
1-month period of checklist use. Other reported 
improvements appear in table 3. 
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Table 2 Implementation studies of the WHO Surgical Safety checklist 



Description of Patient Safety 
Author [year] Practice (PSP) Study design 



Theory or logic model 



Description of organisation 



Safety context 



Sewell ef al 
[201 1] 24 



Helmio ef al 
[201 1 ] 55 



Conley ef al 
[201 1] 28 



Bell and Pontin 
[2010] 56 , Bell 57 

Sparkes and 
Rylah [201 0] 58 

Royal Bolton 
[201 0] 36 



2008 WHO surgical checklist, 
unmodified 



Before and after study, 
comparing pre-training 
period to post-training 



2008 WHO surgical checklist. No 
specialty-related changes, but some 
'minor changes.' Checklist included in 
publication; modifications did not 
exclude any items 



2008 WHO surgical checklist, 
unmodified 



Before and after study 



Case series 



2008 WHO checklist adapted different 
for different surgical specialties. Checklist 
not included in publication 

2008 WHO checklist locally adapted. 
Checklist included in publication; 
modifications did not exclude any items 

2008 WHO checklist, unmodified. Local 
adaptation of it was considered but 
ultimately not done 



Case series 



Case series 



Case series 



'The underlying philosophy of the 
checklist is that a true team approach 
with good communication between 
operating room team members is safer 
and more efficient than a hierarchical 
system that relies on individuals' 



'The idea of the checklist is to be an 
add-on security tool for the defined safety 
standard' 



None explicitly stated 



'Without a doubt, the checklist works 
best when all staff members are engaged' 

Discussed various ways a checklist could 
enhance safety, including teamwork and 
effective communication 

Improve patient safety by enhancing 
teamwork and communication 



A UK hospital, orthopaedic operations. 
28% of operations were urgent, and 
77% involved general anaesthesia 



Finland, otorhinolaryngology head and 
neck surgery ORs. 747 operations in the 
2-month study periods combined. All 
subgroups of otorhinolaryngology head 
and neck surgery were included 



Five Washington State hospitals. Two 
hospitals had <10 ORs, one had 10-20 
and two had >20. Two urban, two 
suburban and one rural 



Large two-hospital trust in the UK with 
10 000 staff and 850 000 patients 
annually 

Teaching hospital in the UK with 29 
ORs in five locations performing 
specialised complex surgery 

Trust in the UK with eight ORs 



Pre-training period February-May 2009 
(480 operations). During this period: correct 
checklist use was 8%, and 47% thought it 
improved team communication; pre-training 
staff perceptions: 55% thought it caused 
an unnecessary time delay, 28% thought it 
improved patient safety, 47% thought it 
improved team communication and 
teamwork, 64% would want the checklist 
used if they were having an operation 

One-month pre-implementation period in 
May 2009 (304 operations): 17% were 
urgent operations; 24% were on children; 
16% were local anaesthesia. Before 
implementation: knowledge of OR-teams' 
names and roles ranged from 61% to 
92%. Discussing risks was 24%. Postop 
instructions recorded 7-84%. Successful 
communication 79-93% 

Nothing reported about pre-existing safety 
culture. The Vice President for Patient 
Safety at the Washington State Hospital 
Association provided 'significant assistance'. 
Checklist introduction December 2008 to 
January 2009. Interviews conducted 
September-December 2009. One of the 
five hospitals had a recent wrong-site 
incision that motivated surgical staff and 
'opened people's eyes to the need for 
ongoing patient safety efforts' 

Nothing about pre-existing safety culture. 
To prepare for the checklist, they set up a 
Patient Safety Working Group 

NR 



Prior to the checklist, the trust already had 
a core group of patient safety experts 
assembled; this group met to discuss how 
to introduce the checklist. They examined 
the previous year's 41 safety incidents and 
all were 'found to be avoidable had the 
checklist been in use' 



Continued 



Table 2 Continued 



Description of Patient Safety 
Author [year] Practice (PSP) Study design 



Theory or logic model 



Description of organisation 



Safety context 



Vats ef al 
[201 0] 26 



Kearns ef al 
[201 1] 25 



Norton and 
Rangel [201 0] 59 



Styer ef al 
[201 1] 29 



Bittle [2011 



Yuan [201 2] 1 



Kasatpibal ef al 
[20 12] 34 



Bohmer ef al 
[2012] 30 



2008 WHO surgical checklist adapted for Case series 
England and Wales. Checklist included in 
publication; modifications did not 
exclude any items 



WHO surgical checklist, version NR. 
Some obstetric-specific checks had been 
added, but the list of revisions was not 
reported. Checklist not included in 
publication 



2008 WHO checklist modified for 
paediatric operations and also to meet 
the 2009 Joint Commission Universal 
Protocol. Checklist included in 
publication. Removed the following three 
items from the WHO checklist: pulse 
oximetry, difficult airway, anticipated 
blood loss 

2008 WHO checklist modified and 
implemented as hospital policy. Selected 
modifications listed. Checklist not 
included in publication 

2008 WHO checklist adapted for 
individual hospital. Checklist not 
included in publication 

2008 WHO checklist modified for local 
practice. Checklist included in publication 



2008 WHO checklist modified and 
translated. Hair removal added to 
checklist. Other modifications not 
described. Checklist not included in 
publication 

2008 WHO checklist modified. Checklist 
included in publication 



Before and after study 



Case series 



Qualitative description 



Qualitative description 



Before and after study 



Case series 



Before and after 



'the checklist ensures that critical tasks 
are carried out and that the team is 
adequately prepared for the operation' 



'Checklists may be used to improve 
patient safety by ensuring that all 
elements of a practice are instituted for 
each new clinical event' 



Checklist can help to reduce breakdowns 
in communication, ineffective teamwork 
and lack of compliance with process 
measures 



Implementing checklist using a PDSA 
cycle stepwise approach leads to 
smoother transition and sustained 
outcomes 

Checklists 'ensure there is adherence to 
proven standards or care' 

Checklists are an inexpensive and feasible 
way to potentially improve quality of 
surgical care in 'resource-limited settings' 



Checklists may reduce preventable 
adverse surgical events, but may be 
difficult or inappropriate to implement in 
a developing country 

Checklists may improve staff's perception 
of patient safety and job satisfaction 



UK academic hospital 



UK study in obstetrics ORs. Tertiary 
referral obstetric centre with ~6400 
deliveries per year 



Children's hospital in the USA 
performing numerous types of paediatric 
surgery 



Teaching hospital in the USA with 44 
ORs 



Large city hospital in New Zealand 



Two hospitals (each with 2 ORs) in 
Monrovia, Liberia. Hospital 1 (1 50-bed 
primary community hospital), hospital 2 
(200-bed, government referral hospital) 



University hospital in northern Thailand 
(1400 beds, 21 877 operations 
annually) 



Institute for research in Operative 
Medicine of the University of Witten/ 
Herdecke 



Nothing reported about pre-existing safety 
culture. Piloted March-September 2008 at 
a London hospital in 58% of operations 
(424/729) among the two ORs selected 
(one for trauma/orthopaedics OR, the other 
for GI/GYN) 

Before introducing the checklist, they 
measured staff attitudes, preserving 
respondent anonymity: 30% 'felt familiar' 
with others in the OR, 81% felt 
communication could improve, 85% felt 
that in elective cases the checklist would be 
useful, 53% felt that in emergency cases 
the checklist would be inconvenient 

At this hospital they had been building a 
quality infrastructure for 5 years prior, and 
had already implemented the Universal 
Protocol 



'This initiative ... was introduced to see 
how the checklist might fit within our 
hospital culture' 

Quality service improvement team 



Liberia is rebuilding health system 
infrastructure after 14 years of conflict. 
Checklist implementation was a 
collaboration with the Ministry of Health 
and Social Welfare in Liberia to characterise 
its impact in low resource context 

Average rate of surgical site infection in 
Thailand is 1.7% 



NR 



Continued 



Table 2 Continued 



Description of Patient Safety 
Author [year] Practice (PSP) Study design 



Theory or logic model 



Description of organisation 



Safety context 



Fourcade ef al 
[2012] 27 



Perez-Guisado 
ef al [201 2] 62 



van Klei ef al 
[2012] 33 

Takala ef al 
[20 11] 63 



Truran ef al 
[201 1] 64 

Vogts ef al 
[201 1] 32 

Askarian ef al 
[201 1] 35 

Levy ef al 
[201 2] 31 

Helmio ef al 
[2012] 15 



2008 WHO checklist modified. Checklist 
included in publication 



2008 WHO checklist. Checklist included 
in publication 



2008 WHO checklist modified. Checklist 
available in online supplementary 
material 

2008 WHO checklist, modified. Checklist 
available in appendix 



2008 WHO checklist, modified. Checklist 
not included 

2008 WHO checklist, modified. Checklist 
included in appendix 

2008 WHO checklist. No modifications 
noted, checklist not included in 
publication 

2008 WHO checklist modified. Modified 
checklist not included in publication 

WHO checklist (unclear if modified). 
Checklist not included in publication 



Case series 

1. Random sample of 
80 surgeries from each 
centre performed over 
18-day interval. 

2. Interviews and 
surveys of participating 
staff 

Descriptive 

cross-sectional study of 
plastics, reconstructive 
surgical procedures 

Before and after 



Before and after 

Before and after 

Case series 
Before and after 

Case series 
Case series 



Checklists may improve surgical 
outcomes, but face barriers to efficient 
implementation 



18 cancer centres in France 



Checklist 'involves new philosophy of 
organisation that is easier to achieve in 
health workers with lower hierarchy' (ie, 
nurses, surgeon residents) 

Checklists enhance teamwork and 
improve handovers decreased avoidable 
errors and complications 

'Checklist would improve awareness of 
safety-related issues and the fluency of 
operations as well as communication 
during surgery' 

The checklist may improve compliance 
with venous thromboembolism 
prophylaxis guidelines 

Checklists 'promote communication and 
teamwork within the OR' 

Checklist may improve patient safety by 
reducing surgical complications 

Low fidelity of checklist execution may be 
a barrier to improving health outcomes 

'This checklist has reduced complications 
and deaths significantly' 



Reina Sofia Hospital (1684 surgeries) 



University Medical centre Utrecht (The 
Netherlands) 

Four university teaching hospitals in 
Finland 



Hospitals in the UK 



The French National Authority for Health 
introduced a modified checklist as 
mandatory. Implemented by French 
National Federation of Cancer Centres 
along with research team from 
Coordination for Measuring Performance 
and Assuring Quality of Hospitals, Institut 
Gustave Roussy 

NR 



Checklist implemented in accordance with 
mandatory policy by the Dutch Health Care 
Inspectorate 

Pilot study to investigate usefulness of the 
checklist in a variety of surgical specialties 
to inform development of a national 
checklist 

NR 



Auckland City Hospital, New Zealand Checklist implemented 2 years prior 



Referral educational hospital in Shiraz, 
southern Iran (374 beds, 6 ORs) 

Academic tertiary care children's hospital 
(Texas, USA) 

Otorhinolaryngology department in four 
Finnish hospitals 



The Iranian Ministry of Health, Treatment 
and Medical Education approved 
nationwide use of checklist in 2009 

Checklist compliance reported at 100%, 
but fidelity of checklist use is unclear 

Checklist implemented in these hospitals 
during WHO pilot project in 2009 



Gl, gastrointestinal; GYN, gynaecology; NR, not reported; OR, operating room; PDSA, plan-do-study-act. 



Table 3 Findings of implementation studies of the WHO Surgical Safety Checklist 



Author/year Training 



Study phases and checklist 
fidelity 



Reasons for success or failure 



Opinions, knowledge and 
behaviour 



Health outcomes 



Sewell ef al 
[201 1] 24 



Helmio ef al 
[201 1] 55 



Checklist forms placed in ORs, 
compulsory training video detailing 
correct and incorrect uses of the 
checklist, emphasis placed on all 
team members being responsible. 
Active discouragement of a simple 
tickbox approach. Checklist training 
was not associated with reductions 
in any complications or mortality 



Training involved a presentation from 
an outside expert and three 45 min 
lectures. Specific guidelines were in 
the OR, and short instructions on 
the back of the checklist 



Conleyefa/ NR 
[201 1] 28 



Training phase first (unreported 
duration). Post-training period June- 
October 2009 (485 operations). 
Correct checklist use 97%: 2 min. 
20% thought it caused an 
unnecessary time delay 



One-month implementation period in 
September 2009 (443 operations) 



Duration of rollout: <2 months at 
three hospitals, >6 months at two 
hospitals 



'The initial implementation of the 
checklist was met with resistance by 
some operating room team members 
as there was a belief that many of 
the points were already in practice' 



'Use of the checklist improved 
verification of patient identity, but this 
was still inadequate.' 'Our study 
confirms that the surgical checklist fits 
well into otolaryngology.' 'We 
recommend the use of this checklist 
in all operations' 



The key is whether the local 
champion can 'persuasively explain 
why and adaptively show how to use 
the checklist.' Implementation was 
incomplete at three hospitals: One 
cancelled attempts to implement the 
checklist due to 'fear of 
insurmountable resistance and poor 
interdisciplinary communication'. 
Another cancelled attempts because 
they were unable to move beyond 
pilot testing. The third had less 
effective implementation because of a 
laissez-faire leadership style; no 
training; staff understood neither why 



77% thought it improved team 
communication, 68% thought it 
improved patient safety, 80% would 
want the checklist used if they were 
having an operation 



'... overall, the operating room 
personnel were supportive'. 
Anaesthesiologists' knowledge about 
patients had improved compared with 
the pre-implementation period. 
Preoperative check of anaesthesia 
equipment increased from 71% to 
84%. After implementation, staff were 
more likely to accurately report patient 
identity, procedure and operative side. 
After implementation, there was 
improvement in: knowledge of 
OR-teams' names and roles ranged 
from 81% to 94%. Discussing risks was 
38%. Postop instructions recorded 
86%. Successful communication 87- 
96% 

Interviews conducted, but no 
quantitative summary of opinions 
provided. Three hospitals were 
discussed in detail 



Early complications 8.5% before 
checklist training and 7.6% after. 
Mortality 1.9% before checklist 
training and 1.6% after. Lower 
respiratory tract infections 2.1% 
before checklist training and 2.5% 
after. Surgical site infection 4.4% 
before checklist training and 3.5% 
after. Unplanned return to OR 1.0% 
before checklist training and 1.0% 
after 

NR 



NR 
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Table 3 Continued 



Author/year Training 



Study phases and checklist 
fidelity 



Reasons for success or failure 



Opinions, knowledge and 
behaviour 



Health outcomes 



nor how the checklist could be 
implemented 

'To implement the checklist 
effectively, it was essential to engage 
all staff to ensure the theatre team 
worked together.' 'Working with 
individuals to identify any gaps or 
issues with implementation.' Currently 
it is 'being used as standard 
throughout theatres' 

Even though people agreed with the 
checklist in theory, it was difficult to 
change attitudes and behaviours, 
particularly the senior team. The 
checklist was required to be signed by 
team members and 'This had led to 
the fear that legal colleagues will 
apportion blame to those who have 
signed the checklist when 
complications occur' 

'The importance of communicating 
with and involving people beyond this 
core group was recognised straight 
away.' 'Essentially it is all about 
changing the culture, which can be a 
long process, but it's well worth it' 



Bell and Pontin 
[2010] 56 , Bell 57 



Sparkes and 
Rylah [201 0] 58 



Royal Bolton 
[2010] 36 



Vats ef al 
[2010] 26 



Training provided to prevent 
'teething problems.' Instead of 
requiring paperwork, they used in 
each OR an A3 board (a drawing 
board about 14x20 inches) that was 
colour-coded to aid completion. 
Publicity campaign in both hospitals 

"Extensive educational support and 
training" 



Drop-in educational sessions which 
involve 120 participants 



Limited time given to training 



Piloted the checklist at one of the two 
hospitals first 



3-month pilot, during which changes 
to the checklist were made. After the 
pilot, and training, the checklist was 
introduced to all 29 ORs in November 
2009 



May and June 2009 were spent 
getting the word out about plans to 
start using the checklist. Piloted first 
for 1 month in two of the Trust's 
hospitals in 62 operations. September 
2009 was the trust-wide launch of the 
checklist. 'Every Trust is different but 
implementing the checklist across the 
Trust rather than a prolonged pilot 
period.' Within the first week 33% of 
operations employed the checklist. By 
1 month it was at 72%. Currently all 
eight ORs use it 

Checklist accelerated with use. Large 
variability in how the checklist was 
used: sometimes incompletely, hurried, 
dismissive replies, and without some 
key participants. Compliance was 
initially good, then fell when the 
research team was absent, and so the 
team had to re-enter ORs to 
encourage greater use. Compliance 



Need a local champion as well as 
local organisational leadership. 
Importance of being able to modify 
to fit local needs, for example, there 
was no need to check pulse oximetry 
because it is already always used 



'Communication and staff morale have 
definitely improved since the checklist 
was implemented' 



Before checklist introduction: 'Although 
all found the checklist to be useful, 
many senior clinicians felt that such 
communication already took place 
informally, and that more paperwork 
would not add to safety.' Audit of 250 
cases in February 2010 found that team 
briefings occurred in 77% and time 
outs in 86% 

'The feedback we received from staff 
was very positive. Most people were 
keen to introduce the checklist as 
quickly as possible' 



Anaesthetists and nurses were 'largely 
supportive'. Some surgeons were 'not 
very enthusiastic'. Awkward 
self-introductions, takes time to achieve 
comfort, steep interpersonal hierarchy, 
ID the patient BEFORE draping, not 
after. Complaints about duplication; 
perhaps a revised checklist could have 
less duplication 



NR 



NR 



1 -month pilot identified nine 
potential incidents that were avoided 
as a result of the checklist 



'At our hospital, we found no 
significant change in overall 
morbidity or mortality, which were 
already very low, after the 
introduction of the checklist. 
However, there was a noticeable 
improvement in safety processes, 
such as timely use of prophylactic 
antibiotics, which rose from 57% to 

Continued 



w 
o 

00 



Table 3 Continued 
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ranged from 42% to 80% in the 
6-month period 

Compliance with the preoperative part 
of the checklist was 61% after 
3 months and 80% after 1 year. 
Compliance with the postoperative 
part of the checklist was 68% after 
3 months and 85% after 1 year 



//% of operations after the checklist 
was introduced' 

NR 



Kearns ef al 
[201 1] 25 



Training, humorous posters provided, 
and 'all staff empowered to remind 
the team to perform the checklist if 
it was forgotten.' 



Norton and 
Rangel [201 0] 59 



3x5 foot posters in each OR. Launch 
involved formal letter to staff, 
electronic training application, 
multiple in-service training sessions, 
and mention in hospital newsletter 



Styer ef al 
[201 1] 29 



Slide presentations, educational 
posters in ORs, one on one sessions, 
frequent email updates 



December 2008 pilot test in six 
paediatric surgical services (general, 
neuro, orthopaedic, otolaryngology, 
plastic surgery, and urology). February 
2009 pilot test on the revised 
procedures, and more minor edits were 
made. 'Go-live' date 1 April 2009 in 
all of the hospital's ORs. Surgical 
chiefs were local champions, and one 
nurse champion was paired with each 
surgeon champion. They divided the 
responsibility for leading the Time Out 
phase among all team members, and 
identified key speaking points. 
Compliance at ORs improved over time 
during this period from July 2009 to 
February 2010 

October 2008, 2-week trial. Day 1: 
checklist used by 2 surgeons; 
anaesthesia/nursing teams recruited to 
participate and provide same day 
feedback. Day 2: feedback 
incorporated, used in 4 ORs, with 8 
surgeons 

December 2008: chiefs of nursing, 
surgery, anaesthesiology and surgical 
services asked to endorse use as 



Authors cited four contributors to 
success: allocation of responsibilities, 
local champion, sense of ownership 
by team members, and ongoing staff 
consultation 



'Use of the Paediatric Surgical Safety 
Checklist encourages multidisciplinary 
teamwork and has brought increased 
communication to our ORs and in 
other areas' 



Early endorsement by executive 
leadership. Each discipline equally 
involved in leading effort. PDSA cycle 
method for gradual implementation. 
Real-time feedback. Each discipline 
should lead a section of checklist. 
Provide data (process and outcome 
measures). Checklist adopted as 
hospital policy 



Staff attitudes 3 months after checklist 
introduction: 50% now 'felt familiar' 
with others in the OR; 70% felt 
communication had improved; 80% felt 
that in elective cases the checklist was 
useful; 30% felt that in emergency 
cases the checklist was inconvenient. 
Fifty-eight patients were asked whether 
they noticed the operating team 
performing a series of checks before the 
operation, and 75% said they did, and 
another 19% remembered it after being 
prompted. Of the combined 94%, they 
all disagreed with the idea that the 
checks would make them worried, and 
93% said they were reassuring 

December 2008 pilot test of 30 
procedures had 80-90% compliance, 
with 'overwhelmingly positive' feedback. 
'Team members have expressed 
satisfaction with the flow and content 
of the checklist' 



NR 



Checklist caught one near miss 
during sign in (site not marked), 
several near misses during time out, 
(antibiotics not given, problems with 
consent forms, site marking not 
visible after draping, missing 
equipment), and sign out (one team 
realised a patient needed straight 
catheterisation, and reviewing 
procedure name helped nurse 
documentation, one specimen was 
incorrectly labelled) 



Allergies: RN added recent new 
allergy to record 

Antibiotics: not given (3), wrong 
antibiotic for procedure (2), surgeon 
changed mind about giving antibiotic 
after confirming procedure, antibiotic 
left in another room 
DVT: scheduled procedure typically 
would not have required compression 
boots, but patient found to have 
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hospital policy 

February 2008: checklist team 
established (leaders from surgery, 
anaesthesia, nursing), project 
manager, administrative fellow 
March 2009: staggered 14-week 
rollout to 44 ORs. Each surgical 
service allotted 4 dedicated weeks of 
attention (week 1: communication, 
week 2: education, week 3: go live, 
week 4: follow-up). During 'go live' 
period, checklist team observer 
assigned to each surgery to educate, 
provide real-time feedback, answer 
questions 

May 2010: 'coaches' from quality 
division assigned to OR to introduce 
checklist, first to plastics, then other 
specialties. Team meetings with coach, 
OR manager, specialty clinical nurse 
manager, head of surgical department 
and senior registrars preceded 
implementation. Feedback regarding 
checklist procedure obtained at 1 and 
3 weeks 

Two months prior and after. All 
patients followed prospectively for 
outcomes and complications until 
discharge or 30 days, whichever came 
first 



history ot DVI 

Safety precautions: heparin drip had 
not been discontinued 
Plan for management of patient: 
chest radiograph after procedure for 
unsuccessful central line placement 
had been forgotten 



Bittle [2011 



Yuan ef al 
[2012] 14 



Kasatpibal ef al 
[20 12] 34 



w 
o 

10 



Quality division 'coaches' educated 
OR teams about checklist, and 
benefits 



Certified registered nurse 
anaesthetists (CRNAs) were 
identified as local leaders of surgical 
teams. CRNAs along with surgeons, 
OR staff participated in 2-week 
training of lectures, written materials 
and direct guidance 
Large printed poster placed in ORs 



Circulating OR nurse participated in 
two meetings and 1-day data 
collection training session 



From March 2009 to August 2009, 
42.6% of operations selected for 
inclusion 

91% of patients confirmed identity, 
site, procedure and gave consent. Only 
19% of surgical sites marked. 
Anaesthesia equipment and 



NR 



Reasons for success: checklist 
implementation catalysed efforts to 
procure equipment (ie, pulse 
oximeter) necessary for safety 
processes 

Reasons for failure include: lack of 
consistent access to crucial resources 
(such as antibiotics, batteries); 
checklist 'did little to change the 
entrenched hierarchy and relationship 
dynamics of OR staff; lack of 
sustained checklist training beyond 
2 weeks 

Compliance with marking of surgical 
site low because: marking materials 
unavailable, procedure was emergent, 
and 'Thai culture' in which 'Thais do 
not make marks on other people, 
especially on the head' 
Also, 'some surgeons assumed that 



Initially 'staff were anxious and 
somewhat apprehensive, but it is now 
an established step in an operation and 
is carried out with confidence' 



'... the checklist's focus on continuous 
improvement helped to foster a shift in 
mind-set among staff who were "just 
used to making it to the end of the 
day" to building a stronger culture of 
safety' 



Surgical teams often did not introduce 
themselves during time out for cultural 
reasons. 'In Thai culture, people usually 
introduce themselves only when they 
first meet someone and are shy about 
publicising their roles' 
Compliance with checklist high for 



Incorrect surgery site pointed out by 
patient 

Reported incidents fell from 12 to 1 1 
compared with reporting period of 
previous year 



Checklist associated with overall 
improved adherence to >4 (out of 6) 
safety processes, decreased surgical 
site infections (AOR 0.28, 95% CI 
0.15 to 0.54), surgical complications 
(AOR 0.45, 95% CI 0.26 to 0.78) 
Stratified analysis revealed improved 
adherence limited to hospital 1 (AOR 
4.06, 95% CI 2.2 to 7.6), decreased 
surgical site infections, surgical 
complications limited to hospital 2 
No improvement in surgical 
outcomes 

NR 
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medication checked in y(J% ot cases. 
Pulse oximeter applied in 95% of 
cases. Allergies, difficulty airway, 
aspiration risk and risk of >500 mL 
blood loss assessed in 100% of cases 



Survey administered before checklist 
implementation, then 12 weeks after 
implementation 



wrong-site surgery would not occur 
because they had not experienced it 
themselves' 

Compliance with hair removal 
procedures was hampered by lack of 
familiarity with proper procedure, lack 
of equipment and requests from 
surgeons 

All participating specialties were 
involved in formulation of the 
questionnaire 

The checklist was modified for 'local 
conditions' based on feedback from 
staff 

Checklist introduced by department 
heads, demonstrating leadership 
Baseline findings and improvement 
after introduction of the checklist 
were presented to staff 

Barriers to success: 

1. Many elements of checklist already 
exist so checklist creates duplication 

2. Poor communication between 
surgeon/anaesthetist 

3. Completing checklist took too 
much time, staff did perceive benefit 

4. Some items confusing because 
they did not fit in with customary OR 
practices (or seemed inappropriately 
timed) 

5. High staff turnover, new staff 
unfamiliar with checklist. 

6. If OR staff not actively engaged 
during checklist, nurses felt concerned 
about 'legal implications of signing 
the checklist as they might be held 
accountable for errors' 

7. Some felt questions were 
repetitive, might frighten patients 
about to undergo anaesthesia 

8. In 5 centres, box for checklist 
could be checked if safety check not 
performed for time constraints. Some 
staff worried this would make 



lite-threatening issues (drug allergies, 
difficult airways, profuse blood loss) and 
confirmation of patient's name, incision 
and procedure. Notably, standards for 
these measures are already current 
hospital policy 

Compliance was low for surgical site 
marking and appropriate hair removal 

OR staff felt that communication culture NR 
in OR was improved, and checklist 
facilitated information about 
intraoperative complications. The 
authors observed there was more 
discussion of critical events between 
surgeons/anaesthesiologists 



Checklist performed in 90.2% of NR 
surgeries. However, checklist was 
completed in only 61% of cases 



Bohmer ef al 
[2012] 30 



NR 



Fourcade ef al 
[2012] 27 



NR 

Training sessions, written materials 
and videos available from the French 
National Authority for Health, but 
use by participating centres was not 
reported 



1 1-29 January 2010. Random sample 
of 80 records from medical record per 
centre were analysed 
Excluded topical anaesthesia, IR, Gl 
endoscopy and CVC placement 
Subsequent interviews with staff and 
surgeons via semi-structured interviews 
and email surveys 
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checklists tail to improve patient 
safety 

Local 10-question checklist already in 
place, containing 8 items from WHO 
checklist 

Checklist completion may be 
necessary for improved health 
outcomes 

Checklist may be less likely to be 
completed in patients undergoing 
emergency surgery who are at higher 
risk of mortality. This raises 
methodological questions of how to 
adjust for patient severity 

NR 



Perez-Guisado 
ef al [201 2] 62 



van Klei ef al 
[20 12] 33 



Takala ef al 
[201 1 ] 63 



Truran ef al 
[20 11] 64 



Vogts ef al 
[201 1 ] 32 



Askarian ef al 
[201 1] 35 



NR 



Information provided in regular 
meetings to OR staff. Posters placed 
in all ORs and electronic systems 



'Brief instructions on the use of the 
checklist were on the checklist 
backside. Written guidelines on how 
to use the checklist were also 
available. Instructions were given in 
order to avoid variation in the use of 
the checklist in different hospitals 
and operating theatres' 



NR 



NR 



Checklist presented to OR head 
Educational packages containing 
checklist and guidelines were 
distributed to surgeons, assistants, 



January-December 2010 
Responsibility for sections of checklist 
was divided between nurses, 
anaesthetists and surgeons 

1 January 2007-30 September 2010 
Checklist implemented 1 April 2009 
Monthly compliance reports provided 
to team managers. OR circulating 
nurses designated in charge of 
checklist completion 



Study initiated in 2009 
Nurses, anaesthetists and surgeons 
surveyed regarding OR practices 
Then, the checklist was implemented 
over 2-4 weeks 

Finally, survey of OR practices repeated 
4-6 weeks after checklist 
implementation 



Checklist introduced April 2009 
Study evaluated compliance with NICE 
venous thromboembolism prophylaxis 
guidelines for 3-week period prior to 
checklist implementation, and 
6 months afterwards 

November-December 2010 
Medical student observed 100 
procedures, documented compliance 



Included all elective general surgeries 
3 months prior to checklist, followed 
by 3 months after implementation 
(144 patients) 



NR 



Authors suggest compliance with 
'sign out' section is low because the 
timing is 'not linked to a specific 
event in patient management' and 
nurses tasked with performing this 
section have many competing 
responsibilities at the end of 
procedure 

NR 



Nurses achieved 99% implementation, 
but surgeons and anaesthetists only 
completed checklists 79% and 72% of 
time, respectively 

Checklist fully completed in 39% of all 
patients. Median number of items 
documented was 16 



Nurses, anaesthetists and surgeons 
reported increased confirmation of 
patient identity and awareness of 
names/roles of team members 
Surgeons reported improvements in 
discussions of critical events with 
anaesthesiologist (34.7-46.2%, 
p<0.001) and gave prescriptions and 
instructions to post-anaesthesia care 
unit more often 

Non-compliance with guidelines for 
venous thromboembolism prophylaxis 
decreased after checklist from 6.9% to 
2.1% 



Compliance with 'sign in' and 'time out' 
sections of checklist was high. However, 
'sign out' was only observed in 2/100 
cases 



Obtaining information for time out and 
sign out sections of checklist improved 
after checklist implemented 



NR 



After implementation, 30-day 
in-house mortality decreased from 
3.13% to 2.85%. Checklist 
associated with decreased odds of 
30-day mortality (AOR 0.85, 95% CI 
0.73 to 0.98) 

Incomplete checklist did not have a 
significant effect on mortality 

Implementation led to discovery of 
systematic error in timing of 
prophylactic antibiotics administration 



NR 



NR 



Surgical complications (before 
discharge) decreased from 22.9% to 
10% after checklist implementation 
Surgical site infections decreased 
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anaesthetists and nurses 
Checklist presented to OR teams 

All OR team members except 
physicians viewed a computer-based 
training presentation one time 
Large poster of checklist placed in 
every OR 



Levy et al 
[2012] 31 



Helmio ef al 
[2012] 15 



Direct observation of randomly 
selected non-emergent surgeries over 
7-week period 



OR staff heard three informative 
lectures before participating in WHO 
pilot study 

Specific guidelines on use of 
checklist were available in the OR 
Brief instructions appeared on the 
back of the checklist 



Checklist implemented in September 
2010. All surgeries (7148) between 
September 2010 and August 2011 
included 

Survey administered October 201 1 



Inadequate education during 
implementation led to confusion 
regarding practical execution of 
checklist. (Unclear if physicians 
received any training) 
Checklist poster in OR lacked practical 
instructions for how checklist should 
be executed, including which team 
members questions are directed 
towards 

Checklist was not adapted for 
paediatric patients and may have 
been less relevant 

Nurses reported 'some senior 
otolaryngologists had negative 
attitudes towards the checklist' 
'Active leadership, regular audits and 
feedback are important for successful 
implementation and maintenance of a 
checklist' 



Although electronic medical record 
reported 100% compliance, only 4/172 
cases completed more than 7 out of 1 3 
checkpoints 

Small post-study survey of OR staff 
revealed confusion about proper timing 
of 'time-out' and team member 
responsible for ensuring checklist 
execution 



Checklist completion rates were: sign in 
62.3%, time out 61.1%, sign out 
53.6% 

76% of OR team agreed checklist 
improved OR safety, 68% agreed it 
improved error prevention, 93% would 
want checklist used during their own 
surgery 

Disregard for checklist use was revealed 
in the open responses: 'answers are 
dismissive', 'it is noisy and staff is not 
concentrating on the checks' ... One 
senior otolaryngologist wrote, 'Time out 
has never been performed in my 
operations'. In addition, there was 
confusion about who should lead each 
check section and when to do checks: 'I 
have never received the information on 
how to use the checklist' 
Positive comments included 'the 
checklist is beneficial', 'it should always 
be used' and 'nowadays no operation 
should be varied out without the 
checklist' 



NR 



NR 



AOR, adjusted odds ratio; NICE, National Institute for Health and Clinical Excellence; NR, not reported; OR, operating room; PDSA, plan-do-study-act. 



Systematic review 



EDUCATION AND COMPLIANCE 

Regarding checklist training, 10 sites mentioned edu- 
cational sessions, seven used posters in the OR, two 
mentioned a hospital-wide publicity campaign, two 
mentioned that training was provided (however no 
details were given), and eight either failed to mention 
training or stated that only limited training was pro- 
vided. Six studies mentioned a pilot testing period; 
these pilot tests lasted 1-3 months and often resulted 
in minor modifications to the checklist. 

Nine studies reported the degree of compliance 
with the checklist; one simply reported 97% compli- 
ance, and two others reported improvement over time 
(from approximately 60% to 80% in one study, and 
from 85% to 95% in another study). Notably, while 
compliance with checklist use was high, the checklists 
were often left incomplete. Fourcade et al 17 reported 
checklist use in 90.2% of surgeries, but completion in 
only 61%. Similarly, Levy et al 31 found that although 
checklist compliance was 100% in the electronic 
medical record, only 4 of 172 checklists completed 
more than 7 out of 13 required checklist items. 
Kasatpibal et al 34 reported that staff had high compli- 
ance with checklist items which had already been 
standard hospital policy, but low compliance for 
checklist items not routinely practiced. 

SURPASS checklist 

Our searches identified no attempts to use the 
SURPASS checklist outside the Netherlands. The 
website (http://www.surpass-checklist.nl/home.isf? 
lang=en) describes a web version of the checklist 
(called SURPASS Digital), which allows one to modify 
the checklist, although the designers of SURPASS 
strongly discourage it (http://www.surpass-checklist.nl/ 
content.jsf Ppageld =FAQ&dang = en) . 

Wrong-site surgery checklists 

We identified four sites describing checklists based on 
the Joint Commission's UP (table 4). The Swiss 
study 37 focused on verifying patient identity and sur- 
gical site. Compared with the first 3 months of imple- 
mentation, the next 3 months saw better compliance 
in checking patient identity and proportion of surgical 
site checks performed. Barriers to implementation 
included surgeons saying they already knew the 
patients or the surgical site was obvious, and the 
failure to include the input of all surgical services in 
developing the protocol. 

The Swedish study 38 involved two hospitals, each of 
which had a recent wrong-site surgery incident, and a 
root-cause analysis suggested that a time-out proced- 
ure might help. A time-out checklist was implemen- 
ted, and 1 year later, a questionnaire showed that 
93% of team members believed the checklist contribu- 
ted to patient safety. 

The English study 39 was conducted at a children's 
hospital in which staff had incorporated an eight-item 



correct-site surgery checklist into an existing surgical 
checklist. Comparing 2008 with 2006, correct com- 
pletion was improved for four of the eight checklist 
items. 

The North Carolina study 37 implemented a check- 
list to prevent wrong-site surgery that was tailored to 
the hospital's preferences and procedures. Staff com- 
mented favourably that they no longer had to remem- 
ber everything on a cumbersome form. 

No implementation advice was found on the Joint 
Commission website or in other published documents. 
In August 2010, the Joint Commission conducted an 
online survey of over 2100 people. 40 The website 
reports high agreement that organisations can fully 
implement the UP, its three steps are appropriate, and 
that 'there is benefit' in using it in the OR, ambulatory 
surgery and hospital units performing invasive proce- 
dures (but the rates of agreement of benefit were 
lower for ambulatory clinics and physician offices). 
The need to modify policies and procedures varied 
greatly across respondents, and no differences were 
found between different types of respondents (eg, 
type of hospital, bed size). 

COSTS 

Costs of implementing a checklist mostly involve 
checklist development and/or modification, formal 
staff notification, training and additional OR time. In 
2010, Semel et al 41 performed a hypothetical decision 
analysis of checklist introduction. The cost was esti- 
mated using the 'opportunity cost of the work that 
would have otherwise been performed by the three 
department checklist champions and the implementa- 
tion coordinator', which was an estimated $12 635 in 
2008 dollars; per-use cost was only $11. But the cost 
of a major surgical complication was estimated at 
$13 372. In the base case, checklist introduction saved 
money. 

Regarding time, Sewell et al 24 reported that 20% of 
staff thought the WHO checklist caused an unneces- 
sary time delay. However, in 2011, Taylor et al 6 
reported that the WHO checklist took only about 
2 min on average. 

ADOPTION AND DIFFUSION 

On 15 May 2013, the WHO's Surgical Safety Web 
Map (http://maps.cga.harvard.edu: 8080/Hospital/) 
indicated that as of 26 March 2012, 4132 hospitals 
had expressed interest in using the checklist and 1790 
of these hospitals have used the checklist in at least 
one operating theatre. 

Many professional organisations have recommended 
adoption of the WHO checklist. These include the 
Institute for Healthcare Improvement (http://www.ihi. 
org), the National Patient Safety Agency in the UK 
(http://www.nrls.npsa.nhs.uk), 42 43 the Canadian Patient 
Safety Institute, 44 45 the Washington State Surgical Care 
and Outcome Assessment Program 46 (http://www.scoap. 
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Table 4 Studies of wrong-site-surgery checklists implementing the Universal Protocol 



Author/year Description of PSP Study design Theory or logic model 



Description of 
organisation 



Safety context 



Implementation details 



Garnerin 
ef al 
[2008] 37 



Verification protocol for 
checking patient 
identity and the site of 
surgery 



Case series 



the prevention of wrong patients 
and wrong site surgery, not to mention 
accountability, demanded an 
intervention aimed at improving the 
way both patient identity and site of 
surgery checks were performed, while 
acquiring the ability to identify and 
correct deficiencies' 



Swiss anaesthesiology 
service located within a 
1200-bed university 
hospital 



Nilsson ef al Preoperative 'time-out' 
[20 10] 38 checklist 



Questionnaire 
after 

implementation 



None explicitly stated 



Two Swedish hospitals, 
bed sizes not reported 



Prior to introduction of the checklist, all 
patients were required to wear ID 
bracelets, and the operative site had to be 
signed by the surgeon. Anaesthesiologists 
were made aware that they were being 
monitored 



In the autumn of 2007, there were two 
incidents of wrong-side surgery at these 
hospitals, and a root-causes analysis 
suggested that a time-out procedure might 
help. The checklist was pre-approved by 
the heads of the operating and 
anaesthesia departments 



Verification protocol developed by an 
interdisciplinary team. It required patients 
to state their identity, comparing the 
statement to the ID bracelet, OR schedule, 
and medical record. Similar types of checks 
for correct site of surgery. Nine consecutive 
months of data were obtained (October 
2003-June 2004), and later 3 subsequent 
months (October 2004, March 2005 and 
October 2005) 

Compared with the first 3 months of 
implementation, the next 3 months saw 
better compliance in checking patient 
identify (63% up to 81%), complete 
compliance with identity checks (10% up 
to 38%), proportion of surgical site checks 
performed (77% up to 93%), and 
complete compliance with surgical site 
checks (32% up to 52%). Compliance was 
stable in subsequent periods 
Authors attributed the improvements to 
increased use of wristbands upon 
admission into the and the use of three 
different sources for verification 
Barriers included surgeons saying they 
already knew that patients or the surgical 
site was obvious, and the failure to develop 
the protocol with the input of all surgical 
services 

Implementation began in December 2007. 
The checklist was a shared responsibility of 
the OR team. One year later, a 
questionnaire was sent to all 704 surgeons, 
anaesthesiologists, operation nurses, 
anaesthetic nurses, and nurse assistants, 
soliciting their opinions about the new 
time-out checklist. Of the 331 responders, 
93% felt that the checklist contributed to 
increased patient safety (either 'without a 
doubt' or 'probably'). When asked about 
eight specific components of the time-out 
checklists, the percentage of respondents 
who felt the component was 'very 
important' varied widely, from a low of 
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Table 4 Continued 



Author/year Description of PSP Study design Theory or logic model 



Description of 
organisation 



Safety context 



Implementation details 



14% tor the introduction ot team members 
to highs of over 80% for patient identity, 
correct procedure, and correct side. 
Regarding the sign-out, 91% felt that the 
item involving the count of surgical 
instruments and sponges was very 
important 

Five people were required to sign the 
documentation: marking surgeon, operating 
surgeon, ward nurse, scrub nurse and 
anaesthetist. Two audit cycles: once in 
2006 (sooner after implementation) and 
once in 2008 (2 years later). Comparing 
2008 with 2006, correct completion of the 
eight items was not at all improved for four 
items (ward nurse signed, operating 
surgeon signed, scrub nurse signed, and 
operating department practitioner signed) 
but was improved for the other four (mark 
site documented, no mark required 
documented, entries legible, and marking 
surgeon signed). 'The lack of 
documentation, of course, may not reflect 
that the new guidance and processes are 
not being followed, but rather that the 
documentation is regarded as a low priority 
part of the process' 

Original checklist in 2005, minor revisions 
for 2006. Demonstrated the checklist 
during educational staff meetings, and new 
staff were given a primer. Staff gave 
positive comments that they no longer had 
to remember everything. The completed 
checklist is kept as part of the medical 
record 



Owers ef al 
[2010] 39 



Anonymous 
2007" 



Correct site surgery 
checklist incorporated 
into an existing surgical 
checklist 



Case series 



None explicitly stated 



English children's 
hospital, bed size not 
reported 



A surgical checklist already existed at this 
facility; they added a correct site surgery 
component 



Checklist to implement 
the Universal Protocol, 
tailored to this 
hospital's preferences 
and procedures 



Case series 



Stated that the checklist provides cues 
for staff when preparing for a 
procedure 



Hospital in North 
Carolina, bed size not 
reported 



Before this checklist, they were using a 
'cumbersome form' to document their 
compliance with the Universal Protocol 



OR, operating room. 



Systematic review 



org), the South Carolina Hospital Association, the 
Spanish Ministry of Health and Spanish Association of 
Surgeons, 47 and the countries of France, 44 Ireland, 48 
Jordan 48 and the Netherlands. 33 Furthermore, several 
organisations in Australia and New Zealand have devel- 
oped modified versions of the WHO checklist: the 
Royal Australasian College of Surgeons, the Australian 
and New Zealand College of Anaesthetists, the Royal 
Australian and New Zealand College of Obstetricians 
and Gynecologists, the Australian College of Operating 
Room Nurses, and the Australian Commission for 
Safety and Quality in Health Care. 49 

Sivathasan and colleagues 50 conducted telephone 
interviews with 238 hospitals in the UK. Almost all 
(99%) of the hospitals had heard of the checklist, and 
its use was already compulsory in 65% of them. In hos- 
pitals where it was not required, 81% used it voluntar- 
ily and 75% planned to make it mandatory in the 
future. Notably, some ORs reported partial use of the 
checklist, that is, intentionally skipping items or skip- 
ping the entire checklist because of time constraints. 

In June 2009, the journal OR Manager received 
online data from 136 subscribers regarding use of the 
WHO checklist. 51 Nearly half (48.5%) reported imple- 
menting the checklist and 64% said the checklist had 
improved safety in the OR. However, 11% of respon- 
dents stated that the checklist was not well accepted by 
surgeons and another 63% said surgeons did accept it 
but 'with reservations.' Nurses were found to have a 
somewhat greater degree of acceptance. 

A 2009 UK survey of 12 oral and maxillofacial con- 
sultants found that all were aware of the WHO check- 
list, but only 5 of 12 were actually using it. 52 Ten of 
12 expressed the belief that it would improve patient 
safety, but 4 of 12 said it would not improve team 
communication. 

Regarding the UP, accredited hospitals are required 
to comply. Therefore the 'diffusion' of the UP is large, 
by mandate. However, as stated earlier, the UP is not 
a checklist. We found no published information on 
how many hospitals actually use a checklist in their 
efforts to comply. 

DISCUSSION 

Several prominent authorities in the field of patient 
safety have promoted checklists in an attempt to 
prevent mistakes related to surgery. Our report demon- 
strates that checklists have been widely adopted, not 
only in Western countries, but in diverse contexts 
throughout the world. Notably, we found evidence 
that checklists are associated with improved health out- 
comes, including decreased surgical complications and 
surgical site infections. Association, however, does not 
imply causation. Thus, we note three important 
caveats. First, checklists are often implemented as part 
of a multifaceted strategy to improve care, which may 
render it difficult to determine whether improvements 
should be attributed to checklists alone or to other 



changes such as improved communication and shifts in 
OR culture. Second, reporting bias may have played a 
role. Eleven out of 21 implementation studies did not 
report health outcomes, potentially due to an absence 
of clear improvements after checklist implementation. 
Third, the reported results do not mean that all surgi- 
cal checklists are beneficial; other surgical checklists 
containing different items may or may not be 
beneficial. 

Many surgical staff have reported favourable atti- 
tudes towards checklist implementation. However, 
numerous implementation issues remain, including 
how to modify a given checklist to a specific hospital 
setting or specific surgical staff. Our report found that 
barriers to effective implementation include confusion 
regarding practical aspects of checklist use, dealing 
with challenges to efficient workflow, obtaining 
regular access to resources and the beliefs and atti- 
tudes of participating staff, particularly surgeons. One 
recurrent theme in the literature on surgical checklists 
is the explicit encouragement of a team-based 
approach. The AHRQ continues to investigate factors 
supportive of effective checklist implementation with 
the 2010-2013 project entitled, 'Factors associated 
with effective implementation of a surgical safety 
checklist'. 53 This project will elucidate how teamwork 
may contribute to the impact of the checklist. 

The WHO checklist's wide adoption and dissemin- 
ation suggests it may serve as a model for policy- 
makers seeking to develop safety strategies in the 
future. This checklist was explicitly designed to be 
modified for widely varying contexts and executed in 
a short time frame to maintain feasibility. The WHO 
website instructs hospitals: 'Do not hesitate to custom- 
ise the checklist for your setting as necessary, but do 
not remove safety steps just because you are unable to 
accomplish them' and emphasises that 'It should take 
no more than a minute to complete each section of 
the checklist' (ie, 3 min in total). 54 The pilot study 
reported that, at various sites, introduction of the 
checklist took only 1 week to 1 month. 5 Checklist 
implementation is relatively inexpensive, with some 
hospitals simply printing posters to be hung on OR 
walls. These practical characteristics of the WHO 
checklist may have significantly promoted its uptake 
and use. Notably, the WHO approach markedly 
differs from that stated by creators of the SURPASS 
checklist, who strongly discouraged its adaption. 
Although SURPASS is more comprehensive, it has not 
been widely implemented, potentially due to the 
resource intensive effort required to track patients 
throughout a surgical hospitalisation. 

In conclusion, the WHO checklist, the SURPASS 
checklist and checklists implementing the Joint 
Commission UP represent promising initiatives with 
suggestive evidence for improving patient safety. Future 
research may clarify the unique nature of their contribu- 
tion and provide insights for effective implementation. 
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Key summary points 



► Surgical checklists such as the WHO Surgical Safety 
Checklist and Surgical Patient Safety System 
(SURPASS) checklist offer a promising intervention for 
decreasing patient morbidity and mortality due to 
surgical operations. 

► The WHO Surgical Safety Checklist has been success- 
fully adapted for implementation in a wide variety of 
settings, including all surgical specialties, academic 
and community hospitals, and industrialised and 
developing countries. 

► Surgical safety checklists were associated with 
increased detection of potential safety hazards, 
decreased surgical complications and improved com- 
munication among operating room staff. Other factors 
independent of checklists, such as concurrent safety 
improvements, may also explain these improvements. 

► Key components of successful checklist implementation 
include enlisting support from institutional leaders, 
training staff on using the checklist, adapting the 
checklist to incorporate staff feedback and avoiding the 
duplication of information already routinely collected. 
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